Method and Apparatus for Image Content Detection and Image Content Replacement System

ABSTRACT

A subject ( 10 ), such as a billboard, has a filtering film ( 15 ) to absorb electromagnetic radiation specifically in a first wavelength band. A detector ( 60 ) provides a first detector signal ( 61   a ) relating to the first wavelength band and a second detector signal ( 61   b ) relating to another, different, second wavelength band, respectively. Suitably, the subject ( 10 ) appears with high intensity in one band and with low intensity in the other. A content replacement unit ( 40 ) produces a mask signal ( 43 ) by identifying regions of contrast between the first and second detector signals ( 61   a   , 61   b ) as target areas ( 75 ). A content substitution unit ( 47 ) selectively replaces the target areas ( 75 ) with alternate image content ( 42 ) to generate modified video images ( 72 ). The system is useful, for example, to generate multiple live television broadcasts each having differing billboard advertisements.

BACKGROUND

1. Field

The present invention relates in general to a system for manipulatingthe content of an image. More particularly, the present inventionrelates to a method and apparatus which detects a target area in one ormore regions of an image, and which replaces the target area withalternate content. In some examples, the present invention relates to adynamic image content replacement method and apparatus suitable for usewith live television broadcasts.

2. Related Art

In the related art, one or more target areas within a video image aredefined and then replaced with alternate images appropriate to specificviewer groups or geographical regions. For example, billboards at aground or arena of a major sporting event are observed as part of atelevision broadcast, and these target areas are electronicallysubstituted by alternate images that are more appropriate for aparticular country or region. In particular, such a system is useful tocreate multiple television feeds each having different electronicallygenerated advertisement content which is tailored according to anintended audience.

WO2001/58147 (Rantalainen) describes a method for modifying televisionvideo images, wherein a billboard or other visible object is identifiedwith non-visible electromagnetic radiation, such as infra-red light.

WO2009/074710 (Rantalainen) describes a further method for modifyingtelevision video images by determining a shared area where the intendedtarget area is overlapped by added graphics (e.g. graphics overlays)with a predetermined graphics percentage of coverage and substitutecontent is added according to the residual percentage of coverage notcovered by the added graphics.

WO2012/143,596 (Suontama) describes a method of detecting which graphicselements, if any, have been added at any given time in frames of a videosignal.

Considering the related art, there is still a difficulty in providing areliable and effective mechanism for detecting a target area within avideo image where content is to be replaced. In particular, there isstill a difficulty in accurately and efficiently defining the targetareas of the images which are to be replaced.

It is now desired to provide an apparatus and method which addressesthese, or other, limitations of the current art, as will be appreciatedfrom the discussion and description herein.

SUMMARY

According to the present invention there is provided a system, apparatusand method as set forth in the appended claims. Other features of theinvention will be apparent from the dependent claims, and thedescription which follows.

In one example there is provided an improved mechanism for detecting asubject within video images, by using a two-channel detector systemwhich applies narrow-band filtering on the subject relevant to only oneof the two channels. As a result, the detector system achieves greatercontrast between the two detected channels and is more accurate whenpicking out a target area corresponding to the subject from thesurrounding area.

In one example system, a subject, such as a billboard, has a filter toabsorb electromagnetic radiation specifically in a first wavelengthband. A detector provides a first detector signal relating to the firstwavelength band and a second detector signal relating to another,different, second wavelength band, respectively. Suitably, the subjectappears with high intensity in the detector signal of one band and withlow intensity in the other. The first and second bands may both beinfra-red wavelength bands. A content replacement unit produces a masksignal by identifying regions of contrast between the first and seconddetector signals. The mask signal may thus identify target areas withinvideo images of the subject. A content substitution unit selectivelyreplaces the target areas with alternate image content. The system isuseful to generate multiple live television broadcasts each havingdiffering billboard advertisements.

In one example there is provided an image content manipulation systemwherein a subject, such as a billboard, has a filter medium applied to asurface thereof. The filter suppresses electromagnetic radiationspecifically in a first wavelength band. A camera observes the subjectto provide video images of a scene. At least one detector unit alsoobserves the scene to derive a first detector signal relating to thefirst wavelength band and a second detector signal relating to adifferent second wavelength band. A mask signal is accurately andefficiently identified by regions of high contrast between the first andsecond detector signals. A content replacement unit selectively replacesone or more target areas within the camera video images with alternateimage content, such as displaying an alternate advertisement on thebillboards.

In one example there is provided a billboard adapted to be used as thesubject within the system discussed herein. The billboard may comprise asubstrate and printed surface or printed sheet, such as for carrying aprinted advertisement image. The billboard may comprise a filter mediumto specifically filter the first wavelength band. The filter may be afiltering film applied to a surface of the billboard.

In one example, the billboard may be printed without using black ink. Inone example, the billboard may be printed with cyan, magenta and yellowinks to form a CMY black. In one example, the billboard is printed so asto maintain absorption characteristics with respect to the first orsecond wavelength bands evenly across the printed surface of thebillboard, even though different areas of the billboard may havedifferent visible colours. In one example, the billboard substantiallyreflects infra-red spectrum wavelengths except for absorbing the firstwavelength band by the filter.

In one example, the billboard may be passively illuminated by reflectingambient radiation. In another example, the subject billboard may beactively illuminated with suitable radiation. In one example, thebillboard comprises a plurality of light emitting units which illuminatethe billboard with radiation. Suitably, the billboard is activelyilluminated at least within the second wavelength band. In one example,the billboard may comprise one or more radiation emitting units whichare arranged to emit radiation, such as infra-red light, to enhance thesecond wavelength band as detected by the second detector signal.

In one example, the billboard may comprise one or more compensatinglight emitting devices which are arranged to emit visible light ofselected wavelengths, in particular to correct for colour defects orimbalances in the appearance of the subject. In one example, one or morelight emitting units are provided together with a selective light filterlayer which restricts an optical output of the light emitting units to adetermined chromatic spectrum. In one example, these compensating lightunits are driven in use to emit light over selected wavelengths, such asblue or near-blue wavelengths, thus resulting in a chromaticallycompensated visible output from the subject billboard. In one example,the compensated visible output may be matched to a substantially flat(even) response characteristic across the visible chromatic spectrum.

In one example there is provided an image content detection apparatus. Acamera image receiving unit receives camera video images observing ascene including a subject, such as a billboard, wherein the subjectcomprises a filter which absorbs a selected first wavelength band. Adetector signal processing unit processes a plurality of detectorsignals relating to the scene including the subject. A first detectorsignal observes the scene in the first wavelength band. A seconddetector signal observes the scene in a different second wavelength bandwhich is distinguishable from the first wavelength band. A mask signalgenerating unit generates and outputs a mask signal that defines areasof the images, e.g. a masking area or target area, corresponding to thesubject according to regions of contrast between the first detectorsignal and the second detector signal.

In one example, the filter on the subject substantially transmits thesecond wavelength band or, conversely, does not absorb to a substantialdegree radiation in the second wavelength band. In one example, thefirst wavelength band and the second wavelength band are mutuallyexclusive. In one example, the first wavelength band and the secondwavelength band are separated by an inter-band gap so as to be readilydistinguished by the detector signals. In one example, the firstwavelength band and the second wavelength band both comprise non-visiblewavelengths. In one example the first wavelength band is closer tovisible wavelength band, comprising wavelengths longer or shorter asappropriate, than the second wavelength band. In one example, the firstwavelength band and the second wavelength band are both infra-redwavelength bands.

In one example, the camera image receiving unit receives the cameraimages which observe the scene including both the subject and anintervening object which at least partially obscures the subject. In oneexample, the object does not specifically filter the first wavelengthband and/or the second wavelength band. Suitably, the intervening objecttreats the first wavelength band and the second wavelength band equallywithout regard to either band. Meanwhile, the subject of interestsubstantially absorbs radiation in the first wavelength band butsubstantially reflects radiation in the second wavelength band. Thus,the system is better able to distinguish between the subject and anyintervening obstacles, such as people, players or equipment.

In one example, the apparatus may further include a content substitutionunit which substitutes the one or more marked target areas in the cameraimages with alternate content according to the mask signal, thereby toproduce and output modified video images. In one example, the contentsubstitution unit performs photo-realistic substitution. The markedareas of the received camera images relating to the subject are suitablyreplaced with the alternate content, thus allowing multiple differentbroadcast feeds appropriate to different audiences.

In one example, the detector signals may be derived from the videoimages captured by the camera, e.g. using visible or near-visible lightradiation capable of being captured optically through the camera, suchthat the camera may also act as the detector unit. In another exampleembodiment, one or more detector units are provided separate to thecamera or cameras.

In one example there is provided a method of detecting a subject in avideo image. The method may include filtering radiation at the subjectto absorb radiation in a selected first wavelength band, providing videoimages of a scene which includes the subject, producing a first detectorsignal observing the scene in the first wavelength band, producing asecond detector signal observing the scene in a different secondwavelength band, generating a mask signal that defines target areas ofthe video images corresponding to the subject according to areas ofcontrast between the first detector signal and the second detectorsignal, and outputting the mask signal.

In one example, the method may further include substituting the markedareas in the video images with alternate content according to the masksignal.

In one example there is provided a tangible non-transient computerreadable medium having recorded thereon instructions which when executedcause a computer to perform the steps of any of the methods definedherein.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show how embodimentsof the same may be carried into effect, reference will now be made, byway of example, to the accompanying diagrammatic drawings in which:

FIG. 1 is a schematic diagram of an example television broadcastingsystem incorporating an example content manipulation system as discussedherein;

FIG. 2 is a further schematic diagram of the example system of FIG. 1;

FIG. 3 is a schematic view of a detector unit as used in the examplesystem;

FIG. 4 is a perspective view of an example billboard as used in theexample system;

FIG. 5 is a perspective view of another example billboard;

FIGS. 6A, 6B and 6C are graphs showing radiation spectra as may beobserved in the example system;

FIG. 7 is a schematic view showing an example content detectionapparatus; and

FIG. 8 is a schematic flow diagram of an example content detectionmethod.

DETAILED DESCRIPTION

The example embodiments will be described with reference to a contentreplacement system, or more generally an apparatus and method for imagecontent manipulation, which may be used to replace content withintelevision video images and particularly to provide photo-realisticreplacement of a billboard for live television broadcasts. However, themethods and apparatus described herein may be applied in many otherspecific implementations, which may involve other forms of video imagesor relate to other subjects of interest, as will be apparent to personsskilled in the art from the teachings herein.

FIG. 1 is a schematic overview of a television broadcasting system inwhich example embodiments may be applied. FIG. 1 includes one or moreobserved subjects 10, one or more cameras 20, a vision mixer 30, acontent replacement unit 40, and a broadcast delivery system 50. It willbe appreciated that the television broadcasting system of FIG. 1 hasbeen simplified for ease of explanation and that many other specificconfigurations will be available to persons skilled in the art.

In the illustrated embodiment, the observed subject of interest is abillboard 10 which carries original content 11 such as an advertisement(in this case the word “Sport”). The billboard 10 and the originalcontent 11 are provided to be seen by persons in the vicinity. Forexample, many billboards are provided at a sporting stadium or arenavisible to spectators present at the event. In one example, thebillboards 10 are provided around a perimeter of a pitch so as to beprominent to spectators in the ground and also in video coverage of theevent.

A television camera 20 observes a scene in a desired field of view toprovide a respective camera feed 21. The field of view may change overtime in order to track a scene of interest. The camera 20 may have afixed location or may be movable (e.g. on a trackway) or may be mobile(e.g. a hand-held camera or gyroscopic stabilised camera). The camera 20may have a fixed lens or zoom lens, and may have local pan and/or tiltmotion. Typically, several cameras 20 are provided to cover the event orscene from different viewpoints, producing a corresponding plurality ofcamera feeds 21.

The billboard 10 may become obscured in the field of view of the camera20 by an intervening object, such as by a ball, person or player 12.Thus, the camera feed 21 obtained by the camera 20 will encounterdifferent conditions at different times during a particular event, suchas (a) the subject billboard moving into or out of the field of view,(b) showing only part of the subject (c) the subject being obscured,wholly or partially, by an obstacle and/or (d) the observed subjectbeing both partially observed and partially obscured. Hence, there is adifficulty in accurately determining the position of the desired subject10 within the relevant video images, and so define a masking area ortarget area where the content within the video images is to be enhancedor modified, such as by being electronically replaced with alternateimage content.

As shown in FIG. 1, the captured camera feeds 21 are provided, whetherdirectly or indirectly via other equipment, to a vision mixing system30, which in this example includes a camera feed selector unit 30 a anda graphics overlay mixer unit 30 b. Typically, the vision mixer 30 islocated in a professional television production environment such as atelevision studio, a cable broadcast facility, a commercial productionfacility, a remote truck or outside broadcast van (‘OB van’) or a linearvideo editing bay.

The vision mixer 30 is typically operated by a vision engineer to selectamongst the camera feeds 21 at each point in time to produce a cleanfeed (CF) 31, also known as a director's cut clean feed. The visionmixing system 30 may incorporate or be coupled to a graphics generatorunit (not shown) which provides a plurality of graphics layers 22 suchas a station logo (“Logo”), a current score (“Score”) and a pop-up orscrolling information bar (“News: story1 story2”). Typically, the one ormore graphics layers 22 are applied over the clean feed 31 to produce arespective dirty feed (DF) 32.

A separate graphics computer system may produce the graphics layers 22,and/or the graphics layers 22 may be produced by components of thevision mixer 30. The graphics layers 22 may be semi-transparent andhence may overlap the observed billboard 10 in the video images. Thegraphics layers 22 may be dynamic, such as a moving logo, updating timeor score information, or a moving information bar. Such dynamic graphicslayers 22 give rise to further complexity in defining the desiredmasking area (target area) at each point in time.

The dirty feed 32 is output to be transmitted as a broadcast feed, e.g.using a downstream broadcast delivery system 50. The feed may bebroadcast live and/or is recorded for transmission later. The feed maybe subject to one or more further image processing stages, or furthermixing stages, in order to generate the relevant broadcast feed, as willbe familiar to those skilled in the art. The broadcast delivery system50 may distribute and deliver the broadcast feed in any suitable formincluding, for example, terrestrial, cable, satellite or Internetdelivery mechanisms to any suitable media playback device including, forexample, televisions, computers or hand-held devices. The broadcast feedmay be broadcast to multiple viewers simultaneously, or may betransmitted to users individually, e.g. as video on demand.

The content replacement unit 40 is arranged to identify relevantportions of video images corresponding to the observed subject ofinterest. That is, the content replacement unit 40 suitably performs acontent detection function to identify target areas or regions withinthe relevant video images which correspond to the subject of interest.The content replacement unit 40 may optionally also suitably perform acontent substitution function to selectively replace the identifiedportions with alternate content, to produce an alternate feed 33 whichmay then be broadcast as desired. In another example, the contentsubstitution function may be performed later by a separate contentsubstitution unit 47 (also called a ‘remote adder’). In which case, anintermediate feed 35 may be carried between an upstream contentdetection unit 40 and the downstream remote adder unit 47, as will beexplained in more detail below.

FIG. 2 shows one example implementation of the television broadcastingsystem in more detail. In this case, the content replacement unit 40receives a suitable video image feed, such as the clean feed 31, andidentifies therein a target area relevant to the billboard 10 as thesubject of interest. The received images 31 may then be modified so thatthe subject of interest 10 is replaced with alternate content 42, toproduce amended output images 41. In this illustrative example, abillboard 10, which originally displayed the word “Sport”, now appearsto display instead the alternate content 42, as illustrated by the word“Other”. In this example, the content replacement unit 40 is coupled toreceive the incoming video images 31 from the vision mixer 30 and toreturn the amended video images 41 to the vision mixer 30.

In one example embodiment, the content replacement unit 40 may beprovided in combination with the vision mixer 30. As one example, thecontent replacement unit 40 might be embodied as one or more softwaremodules which execute using hardware of the vision mixer 30 or by usinghardware associated therewith.

In another example embodiment, the content replacement unit 40 may beprovided as a separate and stand-alone piece of equipment, which issuitably connected by appropriate wired or wireless communicationschannels to the other components of the system as discussed herein. Inthis case, the content replacement apparatus 40 may be provided in theimmediate vicinity of the vision mixer 30, or may be located remotely.The content replacement apparatus 40 may receive video images directlyfrom the vision mixer 30, or via one or more intermediate pieces ofequipment. The input video images may be recorded and then processed bythe content replacement apparatus 40 later, and/or the output images maybe recorded and provided to other equipment later.

In the example embodiment of FIG. 2, the content replacement apparatus40 receives the clean feed 31 directly from the vision mixer 30 andproduces a modified clean feed 41 as output. The graphics layers 22 arethen added to these modified video images 41 through the graphicsoverlay unit 30 b to create at least one modified dirty feed 33 readyfor broadcast.

Further background information regarding these example embodiments ofthe content replacement apparatus may be found, for example, inpublished patent applications WO2009/074710 (Rantalainen) andWO2012/143596 (Suontama) the entire content of which is incorporatedherein by reference.

In the example embodiments, a high value is achieved when images of asporting event, such as a football or soccer match, are shown live to alarge audience. The audience may be geographically diverse, e.g.worldwide, and hence it is desirable to create multiple differentalternate broadcast feeds 33 for supply to the broadcasting system 50 tobe delivered in different territories using local delivery broadcaststations 51, e.g. country by country or region by region. In a liveevent, the content replacement apparatus 40 should operate reliably andefficiently, and should cause minimal delay.

In the example embodiments, the alternate content 42 comprises one ormore still images (e.g. JPEG image files) and/or one or more movingimages (e.g. MPEG motion picture files). As another example, thealternate content 42 may comprise three-dimensional objects in a 3Dinterchange format, such as COLLADA, Wavefront .OBJ or Autodesk .3DSfile formats, as will be familiar to those skilled in the art.

The alternate content 42 is suitably prepared in advance and is recordedon a storage medium 49 coupled to the content replacement apparatus 40.Thus, the content replacement apparatus 40 produces one or more outputfeeds 41 where the observed subject 10, in this case the billboard 10,is replaced instead with the alternate content 42. Ideally, the imageswithin the alternate feed 41 should appear photo-realistic, in that theordinary viewer normally would not notice that the subject 10 has beenelectronically modified. Hence, it is important to accurately determinea masking area defining the position of the billboard 10 within thevideo images input to the content replacement apparatus 40. Also, it isimportant to identify accurately when portions of the observed subject10 have been obscured by an intervening object 12 such as a player,referee, etc. Notably, the intervening object or objects may befast-moving and may appear at different distances between the camera 20and the subject 10. Further, it is desirable to produce the alternatefeed 41 containing the alternate content 42 in a way which is moreagreeable for the viewer, and which is less noticeable or obtrusive.Thus, latency and synchronisation need to be considered, as well asaccuracy of image content manipulation.

As shown in FIG. 2, the example content replacement apparatus 40 isarranged to process a plurality of detector signals 61. In one exampleembodiment, the detector signals 61 may be derived from the video imagescaptured by the camera 20, e.g. using visible or near-visible lightradiation capable of being captured optically through the camera 20,wherein the camera 20 acts as a detector 60. In another exampleembodiment, one or more detector units 60 are provided separate to thecameras 20.

The detector signals 61 may be derived from any suitable wavelengthradiation. The wavelengths may be visible or non-visible. In thefollowing example embodiment, the detector signals 61 are derived frominfra-red wavelengths, and the detector signals 61 are infra-red videosignals representing an infra-red scene image. Another exampleembodiment may detect ultra-violet radiation. In one example embodiment,polarised visible or non-visible radiation may be detected. Acombination of different wavelength groups may be used, such as a firstdetector signal derived from any one of infra-red, visible orultra-violet wavelengths and a second detector signal derived from anyone of infra-red, visible or ultra-violet wavelengths.

In the illustrated example embodiment, one or more detectors 60 areassociated with the camera 20. In the example embodiment, each camera 20is co-located with at least one detector 60. The or each detector 60 maysuitably survey a field of view which is at least partially consistentwith the field of view of the camera 20 and so include the observedsubject of interest 10. The detector field of view and the camera fieldof view may be correlated. Thus, the detector signals 61 are suitablycorrelated with the respective camera feed 21. In the exampleembodiment, the detector signals 61 are fed to the content replacementapparatus 40. In the example embodiment, the detector signals 61 arerelayed live to the content replacement apparatus 40. In another exampleembodiment, the detector signals 61 may be recorded into a detectorsignal storage medium 65 to be replayed at the content replacementapparatus 40 at a later time.

FIG. 3 is a schematic view showing an example content manipulationsystem in more detail. In this example, the system uses infra-reddetectors to determine a position of the subject (i.e. the billboards)within the video images.

In particular, FIG. 3 shows the camera 20 and the one or more detectors60 in more detail. In this example, at least one infra-red detector 60is associated with each of the cameras 20, producing one or more streamsof detector signals 61 a, 61 b. As an example, the one or more detectors60 may be narrow-spectrum near infra-red (NIR) cameras. The detector 60may be mounted adjacent to the camera 20 so as to have a field of viewconsistent with the camera 20. Further, in some embodiments, thedetectors 60 may optionally share one or more optical components withthe camera 20.

The detector 60 may be arranged to move with the camera 20, e.g. tofollow the same pan & tilt motions. In the example embodiments, thecameras 20 may provide a telemetry signal 23 which records relevantparameters of the camera, such as the focal length, aperture, motion andposition. In one example, the telemetry signal 23 includes pan and tiltinformation. The telemetry 23 may also include zoom information or zoominformation may be derived from analysing the moving images themselves.The telemetry 23 may be used, directly or indirectly, to calculate orotherwise provide pan, roll, tilt and zoom (PRTZ) information. Thecamera telemetry signal 23 may be passed to the content replacementapparatus 40, directly or via an intermediate storage device orrecording medium 24, in order to provide additional information aboutthe field of view being observed by the camera 20.

In the example embodiment, substantially the same field of view isobserved by first and second detector signals 61 a, 61 b which aresensitive to at least two different and distinguishable wavelengthranges or wavelength bands. As will be discussed below, these multipledetector signals 61 a, 61 b improve detection of a desired masking areain the content replacement apparatus 40. The wavelength bands may takeany suitable non-identical values. The observations in one band may bedistinguished against observations with respect to the other band. Inother words, the bands are separable to a significant extent. In theexample embodiments, the bands are non-overlapping, but overlappingbands may be used in some cases. In the example embodiments, the bandsmay be separated by an unobserved range or inter-band gap which is notpresent in either wavelength band.

The one or more detectors 60 may be narrow-spectrum near infra-red (NIR)cameras. The detectors 60 may produce at least a first detector signal61 a which is sensitive to a first wavelength band, such as 780-810 nm,and a second detector signal 61 b which is sensitive to a secondwavelength band, such as 820-900 nm. The first wavelength band 61 a maycomprise wavelengths which are closer to the visible wavelength bandthan the second wavelength band 61 b. In the case of infra-red, thefirst wavelength band 61 a may comprise wavelengths which are shorterthan the second wavelength band 61 b.

FIG. 4 is a schematic view showing one example embodiment of thebillboard 10 in more detail. In this example, the subject billboard 10comprises a substrate 13 which carries a printed medium 14, such as aprinted sheet, to display a desired printed message or advertisement.The billboard 10 may be passive, being illuminated by ambient radiation(e.g. from natural sunlight or stadium lights) and reflecting theambient radiation toward the camera 20 and detector 60. Further, thebillboard 10 may be arranged to actively radiate by including aplurality of radiation emitting units, such as light emitting diode(LED) packages.

In more detail, the billboard 10 suitably comprises a plurality oflayers including a substrate 13, a printed sheet 14, a filter medium 15and a transparent protective cover 16. The substrate 13 may take anysuitable size, shape and form. In the example embodiment, the substrate13 is substantially planar, rectangular, and has a degree of structuralrigidity sufficient to maintain the physical integrity of the billboard10 when in use. The printed sheet 14 may be relatively thin andflexible, e.g. paper. The billboard 10 may be arranged so that theprinted sheet 14 is readily replaceable, so that displayed advertisementcontent can be easily changed. The cover 16 is suitably transparent andmay protect the appearance of the billboard 10, and particularly theprinted sheet 14, e.g. from wind, rain and the like.

In this example, the filter 15 is provided in the form of a filteringfilm. In the example embodiments, the filtering film 15 allows visiblelight wavelengths to pass without significant interference, whilstabsorbing at least some non-visible wavelengths. Visible light istypically in the range 400 nm-720 nm, which the filtering film 15 passessubstantially unhindered. By contrast, the filtering film 15 is suitablya narrow band filter with high absorption characteristics for a selectedbandwidth of non-visible radiation. In one example, the filtering filmcomprises a near infra red dye, such as a fine organic compoundcomprising platinum and nickel.

In one example embodiment, the filtering film 15 may be adhered to,coated onto or integrated within the protective cover 16. In anotherexample embodiment, the filtering film 15 is provided behind the printedsheet 14. In this case, the filtering film 15 may be adhered to, coatedonto or integrated within the substrate 13 and/or on a rear surface ofthe printed sheet 14. Conveniently, the printed inks of the sheet 14 areselected to be substantially transparent to the detected wavelengths ofinterest for the detector signals 61. Meanwhile a reflective surface,e.g. the substrate 13 as a white background plate, may reflect across abroad spectrum of interest. Thus, the filtering film 15 may be hiddenfrom view behind the printed sheet 14, which improves an aestheticappearance of the billboard 10 while maintaining the desiredfunctionality. In this example, the billboard 10 is passive, beingilluminated by ambient light and reflecting the ambient radiation towardthe camera 20 and detector 60.

FIG. 5 shows a further example of the billboard 10, which may beactively illuminated by including a plurality of light units 17, such aslight emitting diode (LED) packages. A lens unit and/or a diffuser (notshown) may be provided to distribute light from the LED units 17 evenlyacross an illuminated area of the billboard. Thus, in the exampleembodiment, the light units 17 may form a light box to illuminate theprinted sheet 14 from behind. In particular, the LED units 17 may bearranged to emit infra-red light to enhance the second wavelength bandas detected by the second detector signal 61 b, such the range 820-900nm as noted above.

FIGS. 6A and 6B are graphs which show example radiation spectra as maybe observed in the example content replacement system described herein.

FIG. 6A provides a comparison of an observed radiation spectrum with,and without, the filtering film 15, demonstrating relativelywell-defined narrow band absorption over a first specific wavelengthband in the near infra-red region (Band 1), while not significantlyabsorbing a second wavelength band (Band 2).

Firstly, a white measurement M1 is shown, for comparison, against anarbitrary intensity scale. In this case, the billboard 10 is observedwith a plain white unprinted reflective surface by a broad-spectrumdetector across non-visible infra-red wavelengths such as from 780 to940 nm. This white measurement shows that the detected infra-reddeclines as wavelength increases, which can be explained by the naturalspectrum of ambient infra-red light (e.g. from the sun and/or fromstadium lights), or due to the declining sensitivity of the detectorunits 60 with increasing wavelength.

A filtered measurement M2 is also shown. In this case, the filteringfilm 15 demonstrates significant absorption in the short IR range ofBand 1, while not significantly affecting the longer IR wavelengths ofBand 2. As a result, the filtering film 15 causes the detector signals61 to show the billboard 10 as a dark object in the first detectorsignal 61 a relating to Band 1, but as a light object in the seconddetector signal 61 b relating to Band 2. This enhanced contrast betweenthe two detector signals 61 a, 61 b allows the area of the billboard 10within the field of view to be detected with greater accuracy.

FIG. 6A further shows an observation M3 where the printed sheet 14 isprepared with black (K) ink as with a four-colour printing process(CMYK). In this case, the black ink K significantly absorbs infra-redwavelengths consistently across a wide frequency range, which wouldnegate the observations by the detector 60. Thus, in the exampleembodiments, the printed sheet 14 is prepared using only coloured inksand without any K (black) ink. In the example embodiments, the printedsheet 14 is printed with a three-colour ink printing system using onlyCMY (cyan, magenta, yellow) printing inks as appropriate for printinglarge-scale advertising media. In which case, the response with theprinted sheet 14 is substantially as shown in the observation M2.

A near-black may be achieved by printing only with C, M and Y, which issufficient to maintain the desired visual appearance of the printedsheet 14 when viewing the billboard 10. However, it has been found thatCMY printing allows the billboards 10 to be used more effectively withthe content replacement apparatus 40 discussed herein. For example, thesystem achieves better performance in distinguishing between the subjectbillboard 10 and an intervening dark-coloured obstacle 12 which absorbsinfra-red, such as a person wearing black clothing.

It has been found that absorbing the shorter wavelengths closer to thevisible spectrum advantageously enhances a natural tendency for somematerials used in construction of the billboard, and some printing inksor paints, to absorb in this wavelength band. Hence, deliberatelyabsorbing the shorter band (Band 1), while making the longer band (Band2) appear as bright as possible, reduces any negative impact ofunintended absorbance in the shorter band (Band 1). It is surprisingthat the materials, paint and inks used to construct some of the exampleembodiments of the billboard actually do not absorb the longer band(Band 2) and thus show up as ‘white’ (high intensity) in the seconddetector signals 61 b. Even so, it will be appreciated that the secondwavelength band (Band 2) may suitably lie in the near infra red rangeand still relatively close to the visible spectrum. This allows manypractical benefits, e.g. that lenses intended for optical use will alsobe applicable in this near infra-red region.

In a further alternative embodiment, the system may be configured withthe longer wavelength band being absorbed, while the shorter wavelengthband is not intentionally absorbed. In some practical circumstances,depending upon the materials selected for the billboard and the types ofobstacles encountered, it may be preferable to intentionally absorb thelonger wavelength band in this way. Again, it is surprising that such aconfiguration has been found beneficial.

The system may be configurable to operate in multiple modes, i.e.sometimes with subjects that absorb one (e.g. shorter) wavelength bandand sometimes with subjects which absorb another (e.g. longer)wavelength band.

FIG. 6B illustrates a further measured spectrum M4 as may be observed inthe case of the actively illuminated billboard 10 (e.g. as shown in FIG.5). In this example, the LEDs 17 of the active billboard 10 may bechosen to emit infrared light specifically in the second wavelength band(Band 2), thus further enhancing contrast against the filtered firstwavelength band (Band 1). Thus, it will be appreciated that activelyemitting infra-red radiation from the subject of interest isadvantageous in many practical circumstances.

FIG. 6C is a graph which shows a further illustrative radiation spectrumM5 as may be observed in the example content replacement systemdescribed herein. The measured spectrum M5 which observes the subject 10(billboard) across the visible light spectrum and into near infra-red.Here, it has been noted that areas of the printed sheet 14 which areintended to appear white may instead appear off-white, cream ornear-yellow. It has been observed that there is a tendency for bluewavelengths (e.g. near 430 nm) to be absorbed by the construction of thebillboard, and in particular due to some example embodiments of thefiltering film 15.

In a further enhancement, the subject 10 (billboard) may be providedwith one or more compensating light emitting devices which are arrangedto emit light of selected wavelengths, in particular to correct forcolour defects or imbalances in the appearance of the printed sheet 14.In one example, one or more of the LED units 17 as shown in FIG. 5 areprovided together with a light filter layer which restricts the opticaloutput to a desired chromatic spectrum. Thus, considering the measuredspectrum M5 in FIG. 6C, these compensating light units 17 are in usedriven to emit light over selected wavelengths (in this case near-blue)thus resulting in a compensated output which is nearer to a desired flatresponse.

In the example embodiments, the billboard 10 includes light emittingunits 17 at least some of which are arranged to emit infra-red radiationin the second wavelength band as discussed above. Thus, the system hasbetter performance for a television audience who see the alternate imagecontent applied to the billboards. Further, at least some of the lightemitting units 17 within the billboard 10 are arranged to providechromatic compensation to enhance their visible appearance forspectators locally (and likewise for a local television audience who seethe original images of the billboards).

FIG. 7 is a schematic view showing the example content replacementapparatus in more detail. In the example embodiment, the contentreplacement apparatus 40 comprises a camera image receiving unit 44, adetector signal processing unit 45, a mask signal generating unit 46,and a content substitution unit 47.

The camera image receiving unit 44 receives any suitable video images71. As described above, the camera feeds 21 may be multiplexed togetherto provide a clean feed 31 comprising moving images from differentcameras 20 at different points in time. The clean feed 31 may bemodified with additional graphics layers to produce a dirty feed 32. Oneor more of the camera image feed 21, the clean feed CF 31 and/or thedirty feed DF 32 discussed above, and/or other suitable video imagesignals, may be provided to the content replacement apparatus 40,depending upon the nature of the installation.

The detector signal processing unit 45 suitably receives input signalswhich allow the subject billboards 10 to be identified within the videoimages 71. In particular, the detector signal processing unit 45 mayprocess the infra-red detector signals 61 and/or the camera telemetrysignals 23 discussed above. As noted above, the one or more detectorunits 60 produce a pair of detector signals 61 a and 61 b according torespective first and second wavelength bands. It will be appreciatedthat the filtering film 15 applied to the billboard 10 absorbswavelengths which are consistent with the first wavelength band of thefirst detector signal 61 a, but not the second wavelength band of thesecond detector signal 61 b. Hence, the first detector signal 61 arepresents the billboard 10 as being a relatively dark area, becauseinfra-red radiation in the studied band has been absorbed by thefiltering film 15. Meanwhile, the second detector signal 61 b isrelatively unaffected by the filtering film 15 and hence the subjectbillboard 10 appears as a relatively bright object.

The detector signals 61 a, 61 b relate to a scene or field of viewincluding the subject billboard 10. The scene observed in the detectorsignals 61 a, 61 b is consistent with the scene in the camera images 21from the camera 20. In the example embodiments, the detector signalprocessing unit 45 processes the detector signals 61 a, 61 b,particularly so that the detector signals 61 a, 61 b are correlated witheach other and with the camera feed 21. Further, in the exampleembodiments, the first and second detector signals 61 a, 61 b are timesynchronised with each other and with the video images of the receivedcamera feed 21. The detector signals 61 are preferably digital, or aredigitized by analogue-digital conversion, thereby representing the fieldof view as an array of pixel values. As noted above, in the exampleembodiments the detector signals 61 are based on infra-red wavelengthsand thus represent an intensity of the selected infra-red wavelengths ateach pixel value.

The mask signal generating unit 46 generates a mask signal 43 to beapplied to the video images 71. The mask signal 43 is suitably derivedfrom and is based on the detector signals 61 a, 61 b. In particular, themask signal 43 can be obtained with reference to the difference betweenthe pixel values of the first and second detector signals 61 a, 61 b.The difference value can be compared against a threshold. That is, whenthe difference is greater than a threshold then the relevant pixel orgroup of pixels are determined to be within the target area 75 (shown inblack), whereas when the difference is less than the threshold then therelevant pixels or not part of the target area (i.e. a non-target area76 shown in white for illustration). Notably, there is an enhancedcontrast between the first and second detector signals 61 a, 61 b in theregion of interest relating to the subject 10, which allows the positionor target area of the subject billboard 10 within the image field to beestablished with a high degree of certainty and accuracy. As a result,the mask signal 43 produced by the content replacement apparatus 40accurately defines the desired masking area (target area) 75, even ifthe subject billboard 10 is partially obscured by an intervening object12.

The masking area signal 43 is itself a useful product of the system andcan be output or recorded in a durable storage unit 48 to be used later.In one example embodiment, the content replacement apparatus 40 may beused only to produce the masking area signal 43, and the contentsubstitution operation may be performed downstream by another piece ofequipment. For example, looking again at FIG. 1, the masking area signal43 may be transmitted to the broadcasting system 50 to be carriedalongside the broadcast feed to a downstream content substitution unit47 (remote adder) to insert the alternate content 42 locally prior totransmission by a local transmitter unit 51.

In the example embodiments, the content substitution unit 47electronically substitutes the target area with the alternate content 42to produce the respective alternate feed 41. In the illustratedembodiment, the content substitution unit 47 electronically substitutesone or more of the target areas within the video images 21 with thealternate image content 42 according to the masking signal 43. Thus, thecontent substitution unit 47 in use produces the respective alternatevideo image feed 72.

FIG. 8 is a schematic flow diagram illustrating an example method ofcontent detection. Step 801 includes filtering radiation from a subjectto absorb radiation only in a selected first wavelength band. Step 802includes providing camera images of a scene which includes the subject.Step 803 comprises producing a first detector signal observing the scenein the first wavelength band, wherein the radiation from the subject hasbeen filtered to absorb the selected first wavelength band. Step 804includes producing a second detector signal observing the scene in asecond wavelength band, wherein the second wavelength band does notoverlap with the first wavelength band and the subject has not beenfiltered to absorb the second wavelength band. Step 805 comprisesgenerating a mask signal that defines target areas of the camera imagescorresponding to the subject according to areas of contrast between thefirst detector signal and the second detector signal. Step 806 comprisesoutputting the mask signal, e.g. onto the storage medium 48 as anon-transient tangible recording medium. The mask signal 43 is a set ofdata representing the position of the target area or target areas 75within the video images 71. Thus, the mask signal 43 can also beconsidered as target area determining data.

Example embodiments of the method and apparatus as described herein havebeen found to operate internally or externally in a wide variety oftemperature conditions, from freezing cold to hot sunshine, and evenwith atmospheric interference such as water splashes, fog, rain or snow.It has been found that comparing the first and second detector signalssurprisingly improves the ability of the content replacement system tomake a reliable and efficient determination of the masking area signal,e.g. corresponding to a billboard, even in adverse and challengingpractical environments. Hence, dynamic content replacement is performedquickly and reliably, suitable even for live transmissions.

The industrial application of the example embodiments will be clear fromthe discussion herein.

At least some embodiments may be constructed, partially or wholly, usingdedicated special-purpose hardware. Terms such as ‘component’, ‘module’or ‘unit’ used herein may include, but are not limited to, a hardwaredevice, such as a Field Programmable Gate Array (FPGA) or ApplicationSpecific Integrated Circuit (ASIC), which performs certain tasks.Alternatively, elements of the invention may be configured to reside onan addressable hardware storage medium and be configured to execute onone or more processors. Thus, functional elements of the invention mayin some embodiments include, by way of example, components, such assoftware components, object-oriented software components, classcomponents and task components, processes, functions, attributes,procedures, subroutines, segments of program code, drivers, firmware,microcode, circuitry, data, databases, data structures, tables, arrays,and variables. Further, although the example embodiments have beendescribed with reference to the components, modules and units discussedherein, such functional elements may be combined into fewer elements orseparated into additional elements.

Although a few example embodiments have been shown and described, itwill be appreciated by those skilled in the art that various changes andmodifications might be made without departing from the scope of theinvention, as defined in the appended claims.

1. An image content replacement system, comprising: a subject includinga filter which absorbs radiation specifically in a first wavelengthband; at least one detector unit which observes a scene including thesubject to provide a first detector signal relating to the firstwavelength band and a second detector signal relating to a secondwavelength band which is distinct from the first wavelength band; and acontent replacement unit which generates a mask signal derived fromregions of contrast between the first and second detector signals, themask signal identifying one or more target areas within video imagesobserving the subject for replacement with alternate content.
 2. Thesystem of claim 1, wherein the content replacement unit comprises: avideo image receiving unit which receives the video images observing thesubject; a detector signal processing unit which processes the first andsecond detector signals, wherein the first detector signal observes thescene in the first wavelength band and the second detector signalobserves the scene in the second wavelength band; and a mask signalgenerating unit which generates and outputs the mask signal that definesthe target areas of the video images corresponding to the subjectaccording to the regions of contrast between the first detector signaland the second detector signal.
 3. The system of claim 1, wherein thefilter of the subject substantially transmits the second wavelengthband.
 4. The system of claim 1, wherein the first wavelength band andthe second wavelength band are mutually exclusive.
 5. The system ofclaim 1, wherein the first wavelength band and the second wavelengthband are separated by an inter-band gap.
 6. The system of claim 1,wherein the first wavelength band and the second wavelength band bothcomprise non-visible wavelengths and the first wavelength band is closerto a visible wavelength band than the second wavelength band.
 7. Thesystem of claim 1, wherein the first wavelength band and the secondwavelength band are both infra-red wavelength bands.
 8. The system ofclaim 2, wherein: the video image receiving unit is arranged to receivethe video images which observe the scene including both the subject andan intervening object which at least partially obscures the subject, andthe mask signal generating unit is arranged to generate and output themask signal distinguishing between the subject and the interveningobject according to the regions of contrast between the first detectorsignal and the second detector signal.
 9. The system of claim 1, furthercomprising a content substitution unit which substitutes one or more ofthe target areas in the video images with alternate content according tothe mask signal to output modified video images.
 10. (canceled)
 11. Thebillboard of claim 17, wherein the billboard comprises one or morecompensating light emitting devices which are arranged to emit visiblelight of selected wavelengths to correct for chromatic deficiencies in avisible appearance of the billboard.
 12. An image content detectionapparatus, comprising: a video image receiving unit which receives videoimages observing a scene including a subject; a detector signalprocessing unit which processes a plurality of detector signals relatingto the scene including the subject, wherein a first detector signalobserves the scene in a first wavelength band which is absorbed by afilter of the subject and wherein a second detector signal observes thescene in a second wavelength band which is distinguished from the firstwavelength band; and a mask signal generating unit which generates andoutputs a mask signal that defines target areas of the video imagescorresponding to the subject according to regions of contrast betweenthe first detector signal and the second detector signal.
 13. A methodof detecting a subject in video images, the method comprising: filteringradiation at the subject to absorb a selected first wavelength band;providing video images of a scene which includes the subject; producinga first detector signal observing the scene in the first wavelengthband; producing a second detector signal observing the scene in a secondwavelength band different from the first wavelength band; generating amask signal that defines a target area of the video images correspondingto the subject according to regions of contrast between the firstdetector signal and the second detector signal; and outputting the masksignal.
 14. The method of claim 13, further comprising substituting thetarget area in the video images with alternate image content accordingto the mask signal.
 15. (canceled)
 16. A non-transitory computerreadable medium having recorded thereon computer program instructionswhich, when executed by a processor of a content replacement apparatusthat is configured to receive video images of a scene that includes afirst subject that absorbs radiation in a selected first wavelength bandand to receive a first detector signal observing the scene in the firstwavelength band and a second detector signal observing the scene in asecond wavelength band different from the first wavelength band,configures the content replacement apparatus to: generate a mask signalthat defines a target area of the video images corresponding to thesubject according to regions of contrast between the first detectorsignal and the second detector signal; and output the mask signal forapplication to the video images.
 17. A billboard adapted for use with animage content replacement system that is configured to detect absorptionof radiation in a first wavelength band, said billboard comprising: asubstrate for carrying a printed medium having a printed image; and afiltering medium or film which specifically absorbs the first wavelengthband at least over an area of the billboard occupied by the printedimage.
 18. A method of detecting image content comprising: receivingvideo images of a scene that includes a first subject that absorbsradiation in a selected first wavelength band; receiving a firstdetector signal observing the scene in the first wavelength band;receiving a second detector signal observing the scene in a secondwavelength band different from the first wavelength band; generating amask signal that defines a target area of the video images correspondingto the subject according to regions of contrast between the firstdetector signal and the second detector signal; and outputting the masksignal for application to the video images.